A reinforcement learning theory for homeostatic regulation

Reinforcement learning models address animal’s behavioral adaptation to its changing “external” environment, and are based on the assumption that Pavlovian, habitual and goal-directed responses seek to maximize reward acquisition. Negative-feedback models of homeostatic regulation, on the other hand, are concerned with behavioral adaptation in response to the “internal” state of the animal, and assume that animals’ behavioral objective is to minimize deviations of some key physiological variables from their hypothetical setpoints. Building upon the drive-reduction theory of reward, we propose a new analytical framework that integrates learning and regulatory systems, such that the two seemingly unrelated objectives of reward maximization and physiological-stability prove to be identi- cal. The proposed theory shows behavioral adaptation to both internal and external states in a disciplined way. We further show that the proposed framework allows for a unified explanation of some behavioral pattern like motivational sensitivity of different associative learning mechanism, anticipatory responses, interaction among competing motivational systems, and risk aversion

Spectral Statistics of "Cellular" Billiards

For a bounded planar domain $\Omega^0$ whose boundary contains a number of flat pieces $\Gamma_i$ we consider a family of non-symmetric billiards $\Omega$ constructed by patching several copies of $\Omega^0$ along $\Gamma_i$'s. It is demonstrated that the length spectrum of the periodic orbits in $\Omega$ is degenerate with the multiplicities determined by a matrix group $G$. We study the energy spectrum of the corresponding quantum billiard problem in $\Omega$ and show that it can be split in a number of uncorrelated subspectra corresponding to a set of irreducible representations $\alpha$ of $G$. Assuming that the classical dynamics in $\Omega^0$ are chaotic, we derive a semiclassical trace formula for each spectral component and show that their energy level statistics are the same as in standard Random Matrix ensembles. Depending on whether ${\alpha}$ is real, pseudo-real or complex, the spectrum has either Gaussian Orthogonal, Gaussian Symplectic or Gaussian Unitary types of statistics, respectively.Comment: 18 pages, 4 figure

Cocaine Addiction as a Homeostatic Reinforcement Learning Disorder

Drug addiction implicates both reward learning and homeostatic regulation mechanisms of the brain. This has stimulated 2 partially successful theoretical perspectives on addiction. Many important aspects of addiction, however, remain to be explained within a single, unified framework that integrates the 2 mechanisms. Building upon a recently developed homeostatic reinforcement learning theory, the authors focus on a key transition stage of addiction that is well modeled in animals, escalation of drug use, and propose a computational theory of cocaine addiction where cocaine reinforces behavior due to its rapid homeostatic corrective effect, whereas its chronic use induces slow and long-lasting changes in homeostatic setpoint. Simulations show that our new theory accounts for key behavioral and neurobiological features of addiction, most notably, escalation of cocaine use, drug-primed craving and relapse, individual differences underlying dose-response curves, and dopamine D2-receptor downregulation in addicts. The theory also generates unique predictions about cocaine self-administration behavior in rats that are confirmed by new experimental results. Viewing addiction as a homeostatic reinforcement learning disorder coherently explains many behavioral and neurobiological aspects of the transition to cocaine addiction, and suggests a new perspective toward understanding addiction

Homeostatic reinforcement learning for integrating reward collection and physiological stability

Efficient regulation of internal homeostasis and defending it against perturbations requires adaptive behavioral strategies. However, the computational principles mediating the interaction between homeostatic and associative learning processes remain undefined. Here we use a definition of primary rewards, as outcomes fulfilling physiological needs, to build a normative theory showing how learning motivated behaviors may be modulated by internal states. Within this framework, we mathematically prove that seeking rewards is equivalent to the fundamental objective of physiological stability, defining the notion of physiological rationality of behavior. We further suggest a formal basis for temporal discounting of rewards by showing that discounting motivates animals to follow the shortest path in the space of physiological variables toward the desired setpoint. We also explain how animals learn to act predictively to preclude prospective homeostatic challenges, and several other behavioral patterns. Finally, we suggest a computational role for interaction between hypothalamus and the brain reward system

Insecurity for compact surfaces of positive genus

A pair of points in a riemannian manifold $M$ is secure if the geodesics between the points can be blocked by a finite number of point obstacles; otherwise the pair of points is insecure. A manifold is secure if all pairs of points in $M$ are secure. A manifold is insecure if there exists an insecure point pair, and totally insecure if all point pairs are insecure. Compact, flat manifolds are secure. A standing conjecture says that these are the only secure, compact riemannian manifolds. We prove this for surfaces of genus greater than zero. We also prove that a closed surface of genus greater than one with any riemannian metric and a closed surface of genus one with generic metric are totally insecure.Comment: 37 pages, 11 figure

Low rank perturbations and the spectral statistics of pseudointegrable billiards

We present an efficient method to solve Schr\"odinger's equation for perturbations of low rank. In particular, the method allows to calculate the level counting function with very little numerical effort. To illustrate the power of the method, we calculate the number variance for two pseudointegrable quantum billiards: the barrier billiard and the right triangle billiard (smallest angle $\pi/5$). In this way, we obtain precise estimates for the level compressibility in the semiclassical (high energy) limit. In both cases, our results confirm recent theoretical predictions, based on periodic orbit summation.Comment: 4 page